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PREFACE 

The Guide for External Program Review («Gids voor de onderwijsvisitatie') 
presented here replaces the Guide to External Quality Assessment ( Gids 
voor Exteme Kwaliteitszorg" ) published in November 1988. The Guide has 
been rewritten following experience with the visiting comttees be we « 
1988 and 1990. The amendments have been made as a result of the proposals 
put forward in the memorandum Quality Control within the Universities 
{•Kwaliteitszorg birmen de universiteiten* ) which was accepted by the 
General Board (Algemeen Bestuur) of the VSNU (Association of Universities 
in the Netherlands) and the Governing Board (Bestuurli3k Overleg) on 27 
April 1990. 

The protocol remains as it was set down in the Governing Board on 4 
November 1988. An endeavour has been made to make the description of the 
review process and the- explanation of the protocol as clear as possible 
and, where necessary, to make it more precise. 

Pa-t 1 consists of a description of the review process and an explanation 
of" the protocol. The activities of the various parties concerned are 
further discussed: the visiting committee, the faculty and the VSNU. 

In Chapter 1 the external program review is placed in the broader context 
of integrated quality control. In Chapter 2 the characteristics of the 
review system are presented. The hinge linking external and internal 
quality control is the self study. This plays a vital role in the review 
Process. In Chapter 3 the self study is discussed and directions are given 
as to how it should be set up. In Chapter 4 a checklist is presented for 
the implementation of the self study. 

Chapter 5 consists of an explanation of the protocol with reference to what 
the visiting committee is required to do. Chapter 6 discusses further the 
activities expected of the faculty in the context of the program "view. 
Chapter 7 discusses the role of the VSNU, and the bureau of the VSNU, with 
respect to the program reviews. 

In Part II the protocol is presented as it was laid down by the Governing 
Board (Bestuurlijk Overleg) on 4 November 1988, and which serves as tne 
basis for the review system. 

The 'Guide for External Program Review" and the protocol which it subsumes 
are not intended to be a straitjacket into which every review has to be 
forced. They are intended rather as a manual for the parties concerned in 
order to facilitate the smooth running of the review process. Should it be 
necessary, a differentiated approach is possible. Should the use of the 
Guide lead to any comment or critical remarks, it would be appreciated if 
these could be conveyed to the bureau of the VSNU. 

Should there be any questions concerning the review process, contact can 
always be made with drs. A. I. Vroeijenstijn. who operates from the bureau 
of the VSNU as the process coordinator of the external quality control. 



Ouidi for External Progr»» Review 
VSNU, December 1990 



3 



Guide for External Progrw Review 
VSNU, December 1990 



PREFACE 



DESCRIPTION OF THE REVIEW PROCESS AND EXPLANATION OF THE PROTOCOL 

1. INTERNAL AND EXTERNAL QUALITY CONTROL 

2. THE PROGRAM REVIEWS 1988 - 1990 



3- THE SELF STUDY 

3.1 Ala of the Self Study 

3 2 The approach to the internal evaluation 

3.3 Conditions which the $elf study ought to meet 

3.4 Quantitative data 

3.5 Who implements the self study? 



4. A CHECKLIST FOR THE SELF STUDY 



5. THE VISITING COHMITTEE 

5.1 Nomination end eppointment 



5.2 


The task 


littee for the visit 


5.3 


The preparations made by the visiting coma 


5.4 


Formulation of the prograa 




5.5 


The visit 




5.6 


The report 




6. THE FACULTY 




6.1 


Faculty or degree prograe 




6.2 


The coordinator 


the visiting committee 


6.3 


Information necessary for the purposes of 


6.4 


The self study 




6.5 


Preparation for the visit 




6.6 


Student involvement 




6.7 


The visit 




6.8 


The report 




6.9 


The follow up to the visit 




7. THE VSNU 


AND EXTERNAL QUALITY CONTROL 




7.1 


The timetable of the visit 




7.2 


Responsibility of the VSNU 




7.3 


Contact with the Inspectorate 




7.4 


The Appeals Committee 





PROTOCOL AND PROGRESS OF THE EXTERNAL QUALITY CONTROL 

1. THE AIH AND PROGRESS OF THE EXTERNAL QUALTIY CONTROL 

1.1 The aia 

1.2 The process in broad outline 



2. THE VISITING COMMITTEE 

2.1 The task of the visiting committee 

2.2 The constitution of the visiting committee 

2.3 Recommendation end Appointment of the visiting committee 

2.4 The manner of working of the visiting committee 

2.5 The visit 

2.6 The report 

2.7 Costs end compensation for expenses 

2.8 Discharge of the visiting committee 



3. THE FACULTY 

3.1 The coordinator in the faculty 

3.2 Information necessary for the purposes of the visiting commi 

3.3 The self study 



Guide for External Progran Review 
VSNU, December 1990 



S 



THE ASSOCIATION OF UNIVERSITIES IN THE NETHERLANDS •VSNU' AND THE EXTERNAL QUALITY CONTROL 

4.1 Responsibility of the VSNU 

4.2 The Appeals Committee 

4.3 Timetable of the reviews 



Appendix 1: Overview of the reviews 1988 -1990 
Appendix 2: Tables for quantitative data 

Appendix 3; Appointment decision, members of visiting committees 
Appendix 4: Timetable of the reviews 1991 - 1996 
Appendix 5: Publications in the context of Quality control 
References 



ERLC 



Guide for External Program Review 
VSNU, December 1990 



i 



6 



DESCRIPTION 
OF 

THE REVIEW PROCESS 
AND 

EXPLANATION OF THE PROTOCOL 



Guide for External Progrea Review 
VSNU, December 1990 



8 



1. INTERNAL AND EXTERNAL QUALITY CONTROL 

Since the appearance in 1985 of the policy document 'Higher Education: 
Autonomy and Quality" ("HOAK-nota"), the concept of -quality- has been the 
focus of attention. Ap a result the impression could bn obtained that 
•quality control- is a seoarate activity within the universities. In fact 
continuous and structural attention to quality forms a part of general 
management. As such it can be distinguished, but it should not be separa- 
ted, from general management. What is meant by -quality control is 
systematic, structured attention to quality, directed to the maintenance 
and improvement of quality. Quality control is the necessary condition for 
the guarantee of quality (Vroeijenstijn, 1990). 

A number of steps can be distinguished vithin quality control: 

a) quality monitoring : measurement of Quality* 

The progress of the input, the process, and the resulting output can 
be followed by means of a number of standard instruments. It is 
possible to ascertain at any moment what the situation is with respect 
to quality (- the achievement of preformulated objectives and require- 
ments). A continuous system of observation and measurement (registra- 
tion of student progress; pass /fail ratios; graduate unemployment 
data etc) allows a finger to be held on the pulse at all times, and 
further action to be undertaken should the necessity be indicated. 

b) evaluation : self study : strengths / weaknesses analysis . 

The information systematically collected concerning the realization of 
preformulated goals can sometimes lead to a deeper evaluation. Further 
to that, periodic self evaluation should be undertaken in order to 
arrive at a strengths / weaknesses analysis. 



c) quality im provement 

The results of the evaluations and 
should lead to measures which can be 
ty. The bottlenecks and weak points 
eliminated. 



strengths / weaknesses analyses 
taken in order to improve quali- 
which are identified have to be 



d) management on the basis of oualitv 

Although quality control is no end in itself, but should be integrated 
into general management policy, management on the basis of quality can 
be distinguished within the overall process of quality control. The 
term -quality management- is used by some people here. The results of 
the activities above can sometimes lead to management on the basis of 
quality; whether that is a question of the maintenance of the qualita- 
tively good, or the strengthening of the qualitatively weak, measures 
have to be taken. 
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The system of quality control can be represented by the following diagram: 

QUALITY CONTROL SYSTEM 



Objectives / requirements ■ standards / criteria 



Monitoring: 
quality Monitoring 
quality eenureeent 

continuous process 
of 'finger on the 
pulse' 



- Evaluation 

- Self study 

- strengths/weakness 
analysis 

regular process 
informed by results 
of aonitoring 



quality 
iaoroveaent 



■anaoeeent on the 
basis of quality 



Vithin the system of quality control a distinction can be drawn between 
internal and external quality control. 

What is understood by internal quality control is the systematic, struc- 
tured attention which is applied within an institution in order to maintain 
or improve quality (signalling, evaluation, quality management). 

While it is in general clear what is meant by internal quality control, 
that is not always the case for external quality control. In the literature 
we often find that external quality control is linked with activities 
undertaken by the government, the inspectorate, or some other independent 
body. In that event the term 'external' is related to the nature of the 
initiator. In that sense the ministerial exploration committees are 
considered to be external, as would be the visiting committees set up by 
the Minister in 1986 for the purposes of the SKG ("Selectieve Krimp en 
Groei": Selective Cutback and Expansion) operation. This conception of 
external quality control is more appropriate for the time preceding the 
HOAK phase, before there was any talk of a link between autonomy and the 
guarantee of quality. 

The universities have given another connotation to the concept of external 
quality control. Here what is understood is the systematic and structural 
attention to educational quality for which external experts, the visiting 
committees, are recruited. The term "external" is not related here to the 
nature of ti.e initiator, but rather to the fact that experts from outside 
the institution are involved in the quality, control. The recruitment of 
external experts is not however a control from outside; it is rather an 
extension of the internal quality control. The visiting committee is an 
extra instrument in the hands of a university or faculty, on the one hand 
for the analysis of, or reflection upon, their own functioning; and on the 
other hand for external accountability. 
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To date, quality control at national level has been concentrated on 
research within the system of -Voorwaardelijke Financiering" (Conditional 
Financing System), and on the first phase of the educational program within 
the framework of the external review system. There has been a further 
approach made with respect to the guarantee of educational quality through 
the establishment of a Program Recognition Committee. 

Quality control is set up so that the processes within an organization 
which influence quality can be managed in such a way that the quality 
sought can be achieved. This is expressed in the quality of education 
policy which forms part of general management. A coherent system of quality 
control, of which external control with the recruitment of external experts 
forms a part, is related to both the primary and the secondary processes of 
universities: education, research, social service, and management. 

To date the systems concerning the quality control of education and of 
research have operated separately, while within these not all aspects are 
taken into account. The Conditional Financing System, for example, is 
concerned with a part of the research programs, and research management is 
not attended to. Vith education, attention is concentrated on the first 
phase. Only incidental attention has been paid by visiting committees to 
the second phase. 

In order to arrive at a coherent system of educational quality control 
(whether integrated or not), the following points need to be addressed in 
the very near future: 

The approach to the oualitv control of research 

Vithin the VSNU context the universities are at present considering 
the way in which the Conditional Financing System and the assessment 
of research shall proceed. Subjects of discussion are the scope and 
depth of evaluation, the distribution of the disciplines, the desira- 
bility of a more differentiated evaluation and the question as to 
whether all research ought to be evaluated. 

The approach to the oualitv contro l of education 

To date the visiting committees have confined themselves to the first 
phase. In just one case it has been arranged for a visiting committee 
to attend to an element of second phase education, namely the review 
for university teacher training in 1992. Vithin the framework of a 
coherent system it must be considered whether educational provision as 
a whole can or ought to be involved: the second phase programs, the 
training of Ph.D. students (the AIO programs), post initial educatio- 
nal provision, education of researchers. 

The coherence of the oualitv fontrol of research and education 

Signals come regularly from the separate systems of quality control 
that in the assessment of education, research should also be involved, 
and vice versa. It has been suggested to have a self evaluation 
precede external assessment also where research has been concerned. 
Careful coordination if not integration will be necessary, even though 
extensive problems can be anticipated in the actual implementation . 
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f rngram recognition and diplo m a recognition in interaationa', context 

It is of the greatest importance that, with the disappearance of 
national borders in 1992, the universities in the Netherlands will be 
able to compete with those abroad. This means that it is necessary for 
Dutch educational programs and the associated diplomas to be recog- 
nized internationally. Three steps are necessary in order to proceed 
from the recognition of a program to the recognition of a diploma: 

Evaluation of the program prior to its implementation: Program 
Recognition Committee; 

Evaluation of the program following its implementation. A recog- 
nized program does not necessarily result in a diploma of high 
quality. An unsatisfactory realization of the program can frus- 
trate this. It is therefore necessary to assess quality retro- 
spectively: visiting committee. 

The combination of the results of the Program Recognition Com- 
mitee and the evaluation of the visiting committee can eventually 
lead to diploma recognition. 

Coordination of activ ities rela^pd to oualitv control 

An endeavour is being made to build quality control in education into 
a more coordinated system of general retrospective periodic control, 
possibly preceded in come cases by program recognition, and sometimes 
followed up by an exploration committee. 

For this it is necessary to avoid an overlap of quality control with 
the activities of the Minister (the inspectorate) and of other bodies 
(RAWB) . Within the context of the HOOP dialogue, agreement with the 
Minister must be made with this in mind concerning the timetable of 
activities for a certain period of time. 

While discussions concerning the form of integrated quality control are 
taking place the program reviews will proceed as normal. The first cycle of 
these will be completed according to agreements already made. 
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2> the PROGRAM REVIEWS 1988 - 1990 

In April 1986 it was agreed with the minister of Education fc Science that 
the institutions for higher education should take upon themselves a 
flexible, comprehensive and unambiguous system of external quality control. 
In 1987, in order to put that agreement into effect, within the context of 
the VSNU the universities took the initiative of establishing a system of 
external quality assessment. This system is characterized by the following 
features t 

* it is sojffilSMgni&Iv to the internal quality control: the self study is 
the cornerstone of the system; 

* the most important instrument is the committee of external experts; 
the visiting committee; 

* a discipline oriented approach has been chosen. It is not the in- 
stitute which forms the focus of attention, but the discipline, and, 
within the discipline, the degree program; 

* it i s r » date directed to the educat ional provision, with the inten- 
tion that at some time in the future the quality control of research 
and social service will either be incorporated, or will run parallel 
to the system; 

* it is a na tAanal system : that is to say, all the institutions having a 
certain degree psogram are involved; 

* it is periodic : all disciplines / degree programs will be examined 
every six years; 

* it is comprehensive : within six years all degree programs will have 
been examined; there are some 150 degree programs involved, brought 
into 26 review clusters. 

* it is public : that is to say. on conclusion of the evaluation, the 
visiting committee will bring out a report that will be published by 
those who commissioned it - the VSNU. 

The remit and the manner of working of the visiting committee, together 
with the requirements made of the faculties with respect to the self 
evaluation and the self study report, were laid down in the protocol 
established by the Governing Board on 4 November 1988. The protocol and an 
•xplanation of it were included in the Guide To External Quality Assessment 
(•Gids voor Externe Kvaliteitszorg') (VSNU, July 1988). The timetable and 
the distribution of the degree programs among the visiting committees was 
laid down in the Timetable for Program Reviews and the Distribution of the 
degree programs among the Visiting committees ( -Rooster voor onder- 
wijevisitaties en indelif, der studierichtingen naar visitatiecomissie ), 
VSNU, July 1989. 
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In 1988 the protocol and the manner of working vere tested in the trial 
reviews which took place in: 

* history: 

* physics and astronomy, 

* mechinicll* engineering (including maritime technology). 

<stT,re was a trial that was involved, and the faculties could not be 
ejected" to suffer any disadvantage resulting from any untoward mishaps, it 
expected to reporting of the evaluations to the faculties would 

™ S f 8 ,on€identi^ in the event each committee has in fact brought out a 
reM ^ ^ft«St ;.ch discipline as a whole. The VSNU has published these 
"Trap rt ^e^g^^uaUty of Dutch University > Education ('Over de 
kwalJteit van bet Nederlandse universitair onderwxjs ') . VSNU, 10 B8. The 
course of The trial round of review was reported in: The S™ 1 "^™ <Lm 
ll^lr* 'Trill Reviews' CDe evaluatie van het project proefvisxtaties . 
VSKUx988b After some limited adaptation of the protocol and the Guide xt 
vas decided to proceed with the system of external quality control sup- 
Parted by committees of external experts. It was further decided to xnclude 
the trial round directly in the first cycle of program evaluation. 

In 1989 four committees were at work in the second round: 

* geography (social and physical geography, urban and rural planning, 
demography and pre- and protohistory) ; 

* mathematics and computer science; .. 

* non-western languages and cultures. Slavonic, languages and cultures. 
Finno-Ugrian language and culture; 

* industrial design and aeronautics and space technology. 

The reports were published in April 1990. 

For the third round (program reviews in 1990) visiting committees have been 
appointed for: 



biology; 
lav; 

philosophy; 
economics ; 

electrical engineering* 



From the table in appendix 1 it can be seen that in the ^ee "unds 

(1988 - 1990). 50 different degree programs in 87 different racuitxes 
been involved with an program review. 
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3. THE SELF STUDY 

The external quality assessment is no end in itself, but rather an exten- 
sion of the internal quality control. The hinge linking the external and 
the internal quality control is the self study which the faculty is 
required to present to the visiting committee in accordance with article 
3 3 of the protocol. This self study is the record of an internal evalua- 
tion which takes place within the degree program in the period preceding 
the visit of the visiting committee. 

In this chapter the importance of the self study is considered further. 
Suaaestions are put forward concerning the implementation of the evaluation 
and the drawing up of the self study. The manner in which the evaluation is 
carried out .is determined by the faculty itself. There are however some 
rules which need to be adhered to with respect to the form and content of 
the self study. 



3.1 Aim of the self study 

The aim of the self study is threefold s 

a) stimulation of the internal quality control through analysis of 
strengths and weaknesses; 

b) internal preparation for the visit of the visiting committee; 

c) the provision of basic information for the visit of the visiting 
committee. 

The self study which the faculty presents to the visiting committee 
determines to a very large extent the course and the effectiveness of the 
visit. The better the self study, the better will the committee be able to 
carry out its work. Some of the 50 self studies which have been presented 
to date indicate that some faculties concerned have had the unfortunate 
tendency to perceive the self study as a 'public relations' document An 
attempt is made to portray the situation in as flattering a light as 
possible. The committee has then to commence its activities by breaking 
through the facade in order to arrive at the actual state of affairs. 
Another problem that has been encountered with some of the self studies is 
that they remain at the level of description and are insufficiently 
analytical. An evaluation of one's own situation is missing. 

A good self evaluation describes and analyses the degree program concerned 
as accurately as possible, and includes a self evaluation. The work of the 
committee members is then a matter of the discussion of the self evalua- 
tion, setting it against their own impressions concerning the quality of 
education rather than having to embark on a -fact finding' mission of their. 



own. 
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3.2 The approach to the internal evaluation 

It is the faculty which determines how the internal evaluation is carried 
out. On the basis of experience with the reviews that have taken place so 
far however a number of suggestions can be made which can facilitate the 
process: 

the Faculty Board should take responsibility for the presentation of 
the self study; 

the self evaluation and the setting up of the self study should not be 
the work of one single person; 

the evaluation committee should consist of some three to five people, 
under the chair of the coordinator appointed by the faculty; 

a clear timetable should be set up, assuming a total amount of time 
available of some five or six months between the moment of the formal 
announcement and the actual visit. An early start can often be made 
since the timetable of the reviews is made known beforehand. 

the topics which are considered in the self study (see Chapter 4) 
should be distributed among the committee members and each member made 
responsible for the collection of information, the analysis and the 
evaluation of the situation; 

the draft results should be discussed on as large a scale as possible. 
It is not necessary for there to be a consensus concerning the report; 
it I; however necessary for as many people as possible to be aware of 
its contents. 

The objectives of the faculty form the starting point for the self evalua- 
tion. In any adequately set up self evaluation process the following 
questions will be addressed: 

1 What are the objectives? Are they clear, complete, appropriate and 
useable7 Is there (internal) consensus concerning their mterpretr- 
tion? 

2. Are the educational programs related appropriately to the objectives? 
Are programs and services set up with the achievement of the objec- 
tives in mind? Does it appear that they work well? Do any problems 
arise with their implementation? How can any problems which arise be 
resolved? 

3. Are there adequate means available for the implementation of the 
programs and services? 

4. To what extent are the objectives achieved? How can systematically 
collected data relating to the extent to which- objectives are achieved 
be used? What is the significance of the data? 
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The following literature may be referred to concerning the setting up of an 
internal evaluation: 

* Os V. van (1987), fvaluatie i n het hoeer onderwjjs: control op en 
W rhPt«l ne ™n de kwali i-p^t van hoger onderwjjs , Groningen: w °l te "- 
Noordhoff ("Evaluation in higher education: the control and the 
improvement of the quality of higher education") 

* Kells. H.R. (1983), SfTf-Studv Processes: A Guide for Postsecondary 
Institutions. 

New York: Macmillan Publishing Company. 

* Roe E. et al (1986), Brewing Academic Performance: Approaches to 
r ho ' B^luation of P^artme ^* »nd Individuals. St. Lucia: University 
of Queensland Press. 

the general editorship of G.V. Heijnen, T.H. Joostens i A.I. Vroeijen- 
stijn (eds), Groningen: COWOG ("Quality control, guarantee for quality 
in higher education") 

In addition the faculty may draw upon the expertise available within the 
Centre for Research and Development of Higher Education (the RVO centre) 
for the design and setting up of an internal evaluation. Also, the Center 
for Higher Education Policy Studies of the University of Twente regularly 
organizes workshops on the setting up and carrying out of self studies. 

3.3 Conditions which the self study ought to meet 

. The self study is the reporting of the self evaluation. That is to 
say, the self study is not just descriptive, it is also analytical. It 
includes an evaluation of the problems. At the same time an indication 
is given as to how it is thought that problems identified will be 
dealt with. 

The manner in which self evaluations are carried out can vary; also 
the levels whom are to be involved in the discussion of the report 
will differ from one institution to another. Nevertheless, the respon- 
sibility for the self study lies in any case with the Faculty Board. 

- Since it is the self study which is of the most importance for the 
visiting committees, and these committees have to study a number : of 
reports? it is important for all the studies to follow the same 
format. Directions concerning this are given in Chapter A. 

The self study forms the starting point for the discussions between 
the visiting committee and the faculty. This implies that everyone who 
will De concerned in one way or another with the discussions needs to 
be aware of the contents of the self study. 

- The self study should not extend to more than 30 or at a maximum AO 
pages, excluding the appendices. Clear reference to any appendices 
taken up must be made in the main body of the report. 
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3.4 (Quantitative data 



At the time of the trial reviews an attempt was made from the bureau of the 
VSNU to set up a 'fact book" vhith a series of quantitative data referring 
to the degree program. That practice is not continued since it turned out 
to be too difficult to assemble consistent data on a national basis. 
Instead the necessary quantitative data form part of the self study. The 
manner in vhica this data has been presented has unfortunately often left 
much to be desired. The committees continue to complain about the lack of 
comparability of information, while there is a -lear need for the standard- 
ization of such data as student numbers, establishment of personnel, staff/ 
student ratios, succes rates etc. The committees ask urgently for further 
attention to be paid to this matter. 

In order to obtain uniformly presented data a number of tables have been 
taken up in appendix 2, and it is indicated how these should be completed. 
In the checklist, an indication is given after every 'point of attention 
involving quantitative data which needs to be reported according to a 
particular table. It is urgently requested that this should be strictly 
adhered to. 

3.5 Who implements the self study? 

A program review needs to have as broad a scope and as high a level of 
involvement as possible. That means that in most cases several degree 
programs will be involved with one single review. Considering the fact that 
it is the self study and self assessment which form the basis of the 
quality control, for each degree program people will have to carry out an 
internal evaluation and a strengths /weaknesses analysis according to a 
checklist. Where more than one degree program within a single faculty is 
concerned, then it is assumed that for each degree program people will 
carry out a self evaluation. It is certainly advisable to establish with 
the faculty coordinator beforehand which topics should be dealt with at 
faculty level. The faculty coordinator has the responsibility of ensuring 
that ultimately one single report is presented which includes the self 
studies of the various degree programs concerned, preceded by a general 
account from the faculty's perspective. 
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4. A CHECKLIST FOR THE SELF STUDY 

Since there is a close connection between the work of the visiting commit- 
tee and the self evalution, the form and content of the self study is 
closely associated with Che task of the visiting committee (see article 2.1 
of the protocol). 

In order to reduce the activities to no more than is necessary it is 
urgently requested that the arrangement of topics as is given below should 
be followed, and all the aspects listed should be considered. 

In the checklist a number of -points for attention' are given for each 
chapter which can be considered as the questions to be addressed by the 
internal evaluation. If the checklist is followed carefully then it should 
be possible to arrive successfully in broad outline at a good strengths / 
weaknesses analysis. 

The lists of 'point of attention' are not intended to be followed heedles- 
sly they should rather be perceived as a manual to ensure that nothing is 
forgotten. If topics are listed which do not apply to the degree program 
concerned then that can be indicated with a single comment. If there are 
topics which require particular emphasis in a certain degree program, then 
that is also possible. 



CHAPTER It THE ORGANIZATIONAL SITUATION 

The discipline oriented aproach of the external quality control does not 
always run parallel with the organizational position within the university 
structure in which the unit to be visited finds itself; it is therefore 
advisable to commence the self study with a chapter which indicates the 
structure and place taken up by the faculty / degree program within the 
university organization, and which reports the subject teams involved with 
the form the educational program takes. 

How is this structure perceived? Does it produce any obstacles tending to 
reduce quality, or does it work in such a way that the achievement of 
quality is facilitated? 

CHAPTER 2: THE STUDENTS 



The size of the student intake; the total number of students (see 
tables 1 and 2 in appendix 2) 

The characteristics of the student input (sex, age, geographic origin, 
previous education / professional experience) 

What has been the development in the student input in the last few 
years 7 Have tay important shifts been ascertained? 

How are deficiencies in educational preparation avoided? what problems 
have been encountered in pre-university education? How are deficien- 
cies dealt with? 
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whs* is done with respect to recruitment and publicity? 

2£ is the extent of succes rates in the first (propaedeutic) year? 

and In the degree (doctoraal) phase? (See table 3 in appends 2). 



CHAPTER 3: AM OVERVIEW OF THE EDUCATIONAL PROGRAMS 



3.1 General 

• Points for attention' 

Have objectives been formulated for the degree program? *.,««. 
I Sow are the objectives translated into final student 7 

it been laid down what graduates ought to know and be able to do? Are 

students aware of this? . different 

- tfave separate additional objectives been formulated for the different 

. How 8 doeV the degree program present itself in contrast with similar 

. ?TK ^ ~J2S^t -e there any factors which 

. r.r P \-%?en^ ai rrpt objectives in view of changed cir- 

. Have'pUns'been made for the presentation, or the strengthening of the 
presentation of the profile of the degree program? If so. how is it 
thought that that might be done? 

3.2 The First (Propaedeutic) Year 

'Points for attention' 

What is the general design of the first (propaedeutic) year? What 
' subjects are \aught? What is the relative amount of the varxous 
subjects taught in terms of student study hours? 

- Have any requirements been made of the subjects with respect to their 

mutual interaction? , vp«t- 

. Are there any indications of differences between this fxrst year 
program and that encountered at other institutions? , „.. nn<i of 

- Which subjects play a role in the orienting ^" U £ x "^%££ 
the propaedeutic year? What opinions are held of the °J iet *ing . 
selective and referral functions of the propaedeutic year? How could 

. Xaedtutic year satisfy the requirements made of it by the 

. of wh^as^ T r ram^ 

. Have any structural changes been made in the educational program in 

the last few years? 

It is requested that a list of the most important literature drawn on in 
the propaedeutic year should be included as an appendxx. 
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3.3 The Degree ( •Doctoraal") Phase 
'Points for attention * 

How has the degree stage (• doctoraal") leading up to graduation been 
set up? Can a basic doctoraal program be discerned? Vhat is the 
relationship with the graduation options? 
What graduation options can be distinguished? 

How in general terms are the various graduation options designed? 
Which core subjects can be distinguished, and which elective subjects? 
What is the extent of the fixed program? What variety do the electives 
offer? How much free space is there? 

The profiles of the various graduation options with respect to those 
of other institutions; what has led to the present profile of the 
graduation option? Does the profile satisfy expectations? 
What bottlenecks can be identified within the graduation options? 
The regulation of work experience, long essays and master theses. How 
is the responsibility of the faculty arranged for students who gradua- 
te outside the faculty? 



It is requested that for every graduation option a list of the n 
important literature drawn on in each of the core subjects should 
included as an appendix. 



3.4 Other education offered 
'Points for attention' 

Is there any indication of a program for graduates from higher voca- 
tional education (HBO graduates) wishing to enter the university 
program? Are the programs satisfactory? 

What second phase programs can be distinguished (professional or 

teacher training programs)? 

What short courses are available? 

What is the situation with respect to service and contract educational 
provision? Who is responsible for it? Are people satisfied with it? 
(N.B. the service courses which are presented in other faculties will 
be considered in the context of the review there; only if the delivery 
of service courses elsewhere forms a bottleneck for the degree pro- 
gram's own educational provision can it be considered here) 
What is the situation with respect to the education of Ph.D. students 
(AIO education)? What is the throughput to the second phase7 Does the 
level of the throughput fit with expectations? 
Is there provision for post-doctoraal education? 
Is there provision for part time study in the degree program? 
If so, in what ways does the part time course differ from the full 
time program in terms of design and courses on offer? 
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CHAPTER 4: THE EDUCATIONAL PROCESS 



. What instructional aethodes are utilized in the acquisition and 
transfer of knowledge (the size of the lectures, the design of working 
groups! pracScals and so forth)? Are there educational methodes which 
would be preferred but which cannot be used because of the circumstan- 
ces of the situation (for example large numbers of students)? To what 
extent does this influence quality7 

In what ways are computers made use of in the educational program^ 
lis tie program been set up didactically so that it is Possible to 
complete U within the prescribed time without undue difficulty? If 
there are any bottlenecks, where do they occur7 

- Ire there ™y bottlenecks which can be identified, for example in the 
transfer from the highly structured propaedeutic year (or foundation 
program) to the more loosely organized doctoraal phase (or graduation 
program), work experience, disserations? Can factors which delay 
student success be identified? , 

. how are students assessed? a) In what ways (multiple choice, open 
questions)? b) Vhen are they assessed (within or at the ena of a 
course)? Do the questions represent the final student objectives which 
have been laid down? How are resits organized? 

In the appendix there should also be a list of the titles of the last 25 
master theses wlich students have submitted, and which were found to be 
satisfactory, with an indication of the grades awarded. 

CHAPTER 5: THE ORGANIZATION AND MANAGEMENT OF EDUCATION 

5.1 Educational Policy 
'Points for attention* 

- In what ways is educational policy put into practice? What committees 
are involved? Is the manner in which the degree program's committee 
functions found to be satisfactory? 

_ Is policy with respect to the degree program coupled to that of the 
institution? Are they linked up with development plans? 

5.2 Personnel policy 
'Points for attention' 

What is the number of personnel (see table 4 in appendix 2)7 Are there 
a^y probes Z the personnel establishment? Age distribution? Are any 
vacancies difficult to fill? How many of the staff have their doc 
torates (see table 4 in appendix 2)7 ^ a A Bm tn 
. What policy is followed with respect to the allotment of academic 
staff to education and research? Is a conscious choice ma* >°f lec- 
turers for the first (propaedeutic) year? How is that related to the 
teaching load? The student / staff ratio? 
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How much time is spent on service education for other faculties? 

Is there any policy with respect to the allotment of lecturers to 

informal lectures, to supervision of essays / master theses, practi- 

cals and work experience? . 

Vhat role do didactic qualifications or educational activities play in 

the furtherance of one's academic career? 

Are there any factors which can be identified as having a negative 
effect on personnel policy? 

I Student load, atudent progreso. student counselling and advice con- 
cerning continuation of the atudy 

'Points for attention' 

Has any measurement been made in the last few years of the student 

load in either the first (propaedeutic) year or the degree (doctoraal) 

phase? If so, what were the results? How many hours do students expend 

on their studies? . . 

Have the results led to any measures being taken in terms of the 

rearrangement of the educational program and / or student guidance? 

Is student progress registered? How is that carried out? - 

Does the registration of progess lead to the timely signalling of 

problems? When is contact first made with problem cases? 

lead to remedial and / or preventive action with respect to either the 

individual students or to the arrangement of the program? 

Has any research been undertaken into the causes of the overrunning of 

the time prescribed for the completion of courses? If so, what were 

the findings? What measures were taken as a result7 . 

How is the study advice service (informing students of their ability 

to continue tboir studies) set up? Does it function satisfactorily? 

How is student guidance and counselling set up? 

What is done in terms of dissemination of information 

a) for student applicants; 

b) in the course of the study; and 

c) in career preparation? Is this provided at central (faculty) or at 
decentralized (subject team) level? 

.4 The material environment 

'Points for attention' 

Vhat is the situation concerning material educational provision 

(lecture rooms, rooms for practicals etc); do they meet requirements? 

What is the situation concerning laboratories? 

Vhat is the situation concerning libraries? 

Vhat is the situation concerning computer equipment? 

Hov large is the educational budget for the degree program? Vhat 
percentage of the budget is earmarked for material provision? Is it 
possible to say what the average costs per student are? 
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5.5 Interaction between central and faculty levels 



'Points for attention' 

Vhat is the interaction between the faculty and the Executive Board of 
the University with respect to educational provision, in particular 
vith the facilitation of quality? 

How does the educational policy of the faculty relate to the educa- 
tional policy of the institution as a whole? 

Does the faculty profit in any way from provisions related to the 
facilitation of quality which emanate from the central level? 



CHAPTER 6: INTERNAL QUALITY CONTROL 



'Points for attention' 

In what way has internal quality control been put into practice? 
How is a 'finger held on the pulse'? 

Is there systematic evaluation? If so, in what way is it done? How are 
students involved with the evaluation of education? Vhat is done with 
the results? Is any feedback given to the students? 

In what way is the renewal of education attended to? How are educa- 
tional support services arranged? Is there cooperation with the centre 
for the research and development of higher education (RWO centre)? If 
so, in what manner? Vhat is the opinion of the educational support 

services? „ . . - 

In what way is attention given to the professionalization of the 

academic staff (management courses, educational / didactic training)? 



CHAPTER 7: THE GRADUATES 



'Points for attention' 

Have demands been formulated by the employment field which graduates 
must satisfy? How is that attended to? Is there any indication of 
clear professional profiles? Have any changes taken place in the 
course of time? 

Is it known where graduates find employment? 

Vhat is the situation concerning graduate unemployment? 

Are there any contacts with alumni? 

Vhat is the opinion of alumni with respect to the educational program? 
Are there any structural or informal contacts with the employment 
field? 

Vhat policy is there to adjust the educational program to the employ- 
ment market as far as that is possible? 
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APPENDICES 



The appendices need to be restricted in number to those which are most 
strictly necessary. Should any appendices be taken up, then their relevance 
must be made clear where appropriate in the maiiv body of the text. 

In any case however, the following appendices need to be included: 

a) a list of the names of the academic staff with the following data: 

title; name: specialization, responsible for education in 5 

a list of the literature drawn upon in the first (propaedeutic) year; 
a list of the literature drawn upon in the core subjects of the 
graduation options; . 
a list of the last 25 master theses submitted ind found to be satis- 
factory, together with the grades awarded; 

a list of the most important papers concerning the degree program 
(such as educational reports, development plans, evaluation reports, 
educational policy plans etc.) so that the visiting committee may 
peruse these should they wish to do so. 



b) 
c) 



d) 
e) 



Cuide for External Progri* Review 
VSNU. December 1990 



25 



5. THE VISITING COMMITTEE 



5.1 Nomination and appointment 



General 

Following the formal announcement of the review according to article 2.3 of 
the protocol, the appropriate VSNU-Disciplinary Board (D00: •discipline- 
overlegorgaan") under which the degree program falls is asked to put 
forward names of experts who could be appointed to the committee. In order 
for the review to be successful it is essential for the whole discipline to 
have confidence in the committee. 

In establishing the composition of the committee the criteria in article 
2 2 of the protocol need to be taken into account. A visiting committee 
consists of about seven people. This is because of the distribution of the 
activities and the possibility of temporary unavailability of any committee 
member. In the enlisting of experts an effort needs to be made to cover the 
specializations within a subject as fully as possible. This is particularly 
so where a number of (sub) disciplines are combined. The experts are sought 
in part from within the university world and in part from the employment 
field which takes up the graduates and, insofar as that exists, from within 
the professional association. In seeking experts within the academic world 
it is advisable not only to look for emeriti but also to choose experts who 
are still active. A balanced distribution is desirable. Further it is 
important that one of the members (preferably the. chairperson) should have 
specific expertise and experience with respect to the functioning of the 
university. 

Considering that, to date, attention is concentrated solely on the educa- 
tional program, the availability of knowledge concerning educational 
processes and educational design is also of importance. 

It is also assumed in the protocol that the committee should include at 
least one foreign expert in the field. 

The following formation can be considered as a suggestion for the possible 
composition of a visiting committee: 

A chairperson, not working in one of the faculties to be visited. S/he 
does not need to be an expert in the field, but should have the 
confidence of those who are. If possible the chairperson should have 
experience with the university (management) structure and the develop- 
ments that have taken place in the last few years. 

1 expert in the field who has been active in the subject area, but has 
nov retired; 

1 expert, still active in the field, whether or not in the faculties 
to be visited (such a member does not take part in the visit to 
his/her own faculty); 
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2 experts in the field from the employment area taking up graduates 
and/or from the professional association; 

An expert from abroad, either Dutch but working outside the Nether- 
lands, or a foreigner vho is familiar with the Dutch situation. (If it 
is not a question of the expressly chosen international composition of 
a committee, then the foreign committee member must have a command of 
the Dutch language.) 

1 expert in the field of education / learning processes. An expert 
from the area of research and development of higher education may be 
considered, or an expert in the teaching of the subject. 

In case it is the view within the discipline that an international commit- 
tee ought to be set up. then a number of the places listed for subject 
experts may be taken up by foreign experts. 

The secretariat of the committee is taken care of by the Bureau of the 
VSNU. 

proposal Vfr TM-DiBciplinftrv Board U2QQ1 

Bearing these criteria in mind, the VSNU-Disciplinary Board formulates a 
proposal for the Executive Board of the VSNU. The proposal consists of a 
list of at least seven names, and the VSNU-Disciplinary Board proposes a 
chairperson. The chairperson of the VSNU-Disciplinary Board is asked to 
ascertain beforehand the readiness of the proposed committee members to 
serve, without however entering into any commitment. With respect to the 
.uegested chairperson the VSNU-Disciplinary Board takes up contact with 
the VSNU before consulting him/her. The Secretary of the VSNU-Disciplinary 
Board sends the nomination to the Chairperson of the VSNU. The nomination 
consists of the following personalia of the persons concerned; 

- name and title; 

- address and telephone number; 

- a short curriculum vitae. 

In case the people concerned are in principle prepared to accept 

Son, then the names of the first nominees will suffice. Vhere there is any 

doubt a number of reserve names need to be included. 

ThP Execut ive Board CPB") 

The Executive Board of the VSNU considers the proposal of the VSNU- 
Disciplinary Board. Should there be any reason not to take up the proposal 
of the VSNU-Disciplinary Board, the Executive Board contacts the chair- 
per.on of the VSlS-Disciplinary Board. Should the VSNU-Disciplinary Board 
lot be able to put forward a complete proposal within six ^en the 

Executive Board can itself fill the nominations. As soon as the Executive 
Board has set up the nomination list, the chairperson of the VSNU contacts 
the proposed members. 

In order to estimate the time needed for the work of a review it can 
roughly be assumed that 4 or 5 days per arranged ^V 1 ^"*^ 1 
preparatory meetings, reading, reporting etc) are required. After accep- 
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twice by the nembers the committee is convened and notice of the decision 
is sent to the following: 

- the chairperson and members of the visiting committee; 

- the Executive Boards of the universities; 

- the faculty bureaus concerned 5 

- the chairperson of the VSNU-Disciplinary Board concerned; 

- the Inspectorate for Higher Education; 

- the organizations where the various committee members work. 
5.2 The task 

In article 2.1 of the protocol the remit of the committee is formulated as 
follows : 

a) to form an opinion on the basis of information supplied by the faculty 
and by means of discussions held on the spot about the standard of 
education and the quality of the educational process, including the 
organisation of education and the standard of the graduates; in 
assessing quality, the committee must look at the requirements- 
/expectations of the student, the faculty/discipline and society, and 
prospective employers in particular; 

b) to make suggestions on quality improvement. 

a number of terms in this remit require further exposition: 

«... to form an opinion concerning the Quality of the 
provision ..." 



Forming an opinion 

The committee is requested to form an opinion. The concept 'form an 
opinion' ought not to be interpreted as -sitting in judgment and handing 
out a sentence in terms of -good' or 'bad' '. Keither is it a natter of 
approval of an educational program; the committee is not directed to 
accreditation or program recognition. 

Although the committee will certainly make comparisons between different 
degree programs for its own purposes, and in its final report a comparative 
evaluation can be given, it is not the intention that the committee should 
set up a rank order of the faculties visited. 

The aim of the educational visit is rather to follow up through dialo 8" e 
with the faculty the points of strength and of weakness indicated in the 
■elf study. The committee can, as a group of expert outsiders, hold up a 
mirror to the faculty. 

The starting point for the committee is the objectives which the degree 
program has formulated for itself. Here it is impossible to evade the 
question as to whether these objectives conform with those generally 
accepted to be appropriate for a particular program, certainly when the 
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programs concerned have a civil effect, be it on the ground of legal 
prions or in actual fact. Neither will the 8C holar ship of the educa- 
tional program escape attention. Comparisons with courses abroad will also 
have their place. 

The fact that the committee does not pass a sentence is not to say that it 
cannot make critical comments. Obscure, cryptic, or occluded language 
should be avoided in the interests of the credibility of the system. What 
is needed is fair criticism without sentence. 

Quality 

•Oualitv' is a concept vhich eludes simple definition. A fairly useful 
definition of quality is: 'the extent to vhich the pre-formulated objec- 
tives are achieved' (De Groot 1986). Another definition proposes that 
quality is equivalent to 'the extent to which a product satisfies the 
requirements laid dovn' . This rather suggests that the concept q^^y 
takes on a meaning dependent upon whoever lays dovn the objectives or the 
requirements The different interested parties will each define quality for 
Xemselves. in accordance with the objectives they set for themselves 
These can run parallel to each other, but they may be contradictory. The 
quality of an educational program, considered from the point of view of the 
Sent', for example, the employment field, means in the first instance 
the quality of a graduate. A student will however be more inclined to 
consider the quality of the design and delivery of the educational program. 
From the perspective of the discipline attention will primarily be given to 
tne demands made of a graduate and on the expectations of the students; but 
in addition other specific demands will be made - for example, that the 
educational program must keep track of recent developments in the field. In 
fact 'the' quality cannot be discussed, although qualities or aspects of 
quality may be. This means that visiting committees have to be involved 
with a range of quality requirements in their work: 

- what are the requirements of a 'client' in the employment field who 
will take on the graduates: 

- what are the demands made by the students following t>e educational 
program, and what are their expectations; 

and what are the demands made on the educational program by develop- 
ments within the discipline itself? 

Th, ft gf»ndards of Education 

The quality of the educational program is determined by a number of 
factors. In the first place it is strongly dependent on the quality ' of the 
lecturers and the manner in which knowledge is transferred This is 
difficult to Judge at first hand without spending a great deal of time in 
lectures and seminars. Indirectly, an opinion can be formed by drawing on 
the comments of students with respect to the educational provision, the 
extent to which the lecturers are able to receive didactic training, and 
the extent to which didactic qualities are taken into account in the 
appointment and promotion of staff. 
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The quality of the educational program is further determined by the content 
and the level of the subject matter taught. The content is again strongly 
dependent on the objectives, and the manner in which these are translated 
into final student assignments. To a certain extent the course description 
gives some insight into the course content. Questions which can be asked 
concerning this are related to the consistency of the program, the underly- 
ing philosophy (why has this program in fact been chosen?). Are the program 
elements necessary for practice in the professional field included? Does 
the program conform with recent developments in the discipline? Is the 
educational program of a sufficiently high scholastic level? Does the 
content of the educational program satisfy the requirements of occupation 
and profession? In order to form an overall view of the educational level 
of the program the committee members vill have to continue to collect and 
place together the separate pieces of a jig saw puzzle. The necessary 
knowledge can be accumulated from the study of the following: 

the literature lists for the first year (propaedeutic) and degree 

(doctorasl) phases; 

student master theses; 

tests and examination questions; 

course descriptions and readers. 

The experts in the committee are expressly chosen so that, on the basis of 
the available information, they will be able to form a picture of the 
content and level of the educational program in a very short time. 



The educat ional process 

Other than through the direct transfer of knowledge, content and level, the 
quality of the educational program is also determined by the educational 
process. Apart from considering an individual teacher, it can be seen 
whether in general the most appropriate instructional methods are u?ed for 
transfer of knowledge to the students. Has the program been so set up that 
it can be completed by a student in the prescribed time without too many 
difficulties? How has the first year (propaedeutic) been built up? The 
degree (doctoraal) phase7 Are there unnecessary bottlenecks? Does the 
program match up with the reasonable expectations of students 7 



Educational organization a nd educational management 

The given restraints and educational policy also determine the quality of 
the educational program. What are the conditions under which the degree 
program must give shape to the educational process? What is the relation- 
ship between teaching and research? What is the situation concerning the 
study load? What policy is followed with respect to education? An important 
aspect of educational organization and management is the structural quality 
control. In what ways are these attended to? What is done as a result of 
evaluations? 
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The different factors which determine the quality of the educational 
program can be represented as a series of concentric layers: 



educational management 
educational organisation 



educational process 



content 




lecturer 






+ 






S student 




4 content 



3 educational process 



2 educational organisation 

, 

1 educational management 



How deep ought a committee to dig? Taking into account the restraints 
within which the visiting committee has to work, there is conflict between 
on the one hand taking a deep look at the educational provision and getting 
to the core of the matter, and on the other hand looking down from a higher 
level of abstraction (educational process, educational organization, 
educational management). The expectation is that if the first three layers 
are peeled off and these look healthy enough, then it can be assumed that 
the fourth layer is equally sound; and certainly if a glimpse of the fourth 
layer can be seen. Should the first layers display some bad or rotten 
patches, then, even if it is only possible to drill down and take a sample, 
the committee will in any case have to get to the heart of the matter. 

The greater the number of degree programs in the committee -s remit, the 
greater will be the conflict. If there are several degree programs it is 
not possible for a visitxng committee, within the given restraints, to go 
deeply into the content. The committee will have to establish on the basis 
of a meticulous scrutiny of the self study reports which of the degree 
programs need an examination in depth and which can be considered from the 
organization process and management aspects alone. The committee will 
however take a sample now and again and carry out a study in depth in a 
degree program, even though there is no indication that it is necessary. 



o 
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Criteria f pH Points fr.n be Raised 

One of the first questions which can be put concerning visiting committees 
is: "what are the criteria to which they work? 1 

Vith reference to an evaluation of quality there are t,vp questions which 
are of importance: 

The first question is: c»ti quality be Quantified? Opinions are divided. It 
is evident that, where people try to derive quality directly from quantita- 
tive data differences of interpretation will arise. Consider, for example, 
the measurement of the quality of research. Is the total number of publica- 
tions a true measure of quality? The analysis of information and experience 
gained elsewhere indicate that that is not unequivocally the case. Another 
example which refers to education: the interpretation of pass/fail data. Is 
a high success rate an indication of quality, or is it rather an measure of 
reduction of standards? 

The problem concerning the quantification of quality makes it necessary to 
consider quantitative and qualitative data interactively. Further, con- 
clusions cannot be drawn from quantitative data on their own. The results 
must first be the subject of discussion and dialogue. 

The second question is: ran standards be set for quality? It will already 
be clear that it is not quality as such, but rather aspects of quality 
which should be discussed; and, where there are different demands and 
paymasters, it is impossible to speak of one set of norms. There are 
several systems of standards and criteria against which quality has to be 
set. It is evident that quality control must have an anchor point. Stan- 
dards and criteria can be derived from the objectives and/ or the demands 
which are pre-formulated or required by those who pay attention to quality. 
This means that the government, which concerns itself with the testing of 
the calibre of an educational program from the perspective of its objec- 
tives will formulate different standards and criteria than, for example, 
the employers will. And just as the requirements made with respect to the 
quality of education can diverge, so can the standards which are applied to 
measure it. 

The objectives formulated by the faculty form the starting point for the 
visiting committee. Implicitly, academics know very well what good educa- 
tion is, or ought to be, and what it may be expected that a graduate should 
know. Explicitly, however, the criteria need to be reflected in the 
objectives and final assignments which are laid down for an educational 
program. A visiting committee should attempt to make more explicit the 
implicit ideas over what makes up a good educational program and a well 
educated graduate. 

Although a list of standards and criteria may not be available beforehand, 
it can nevertheless be .aid quite clearly what a committee ought to 
consider. From experience with reviews to date it is evident that the 
following topics require attention: 

the objectives and final assignments and the translation of these into 
the curriculum; 
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the articulation of secondary school / university education and the 
problem of deficiencies; 

the selection and orientation functions of the propaedeutic year. How 
do these work? what is the situation concerning the advice function 
related to continuing / discontinuing the study after the propaedeutic 
year? 

the transfer from the (frequently strictly organized) propaedeutic 
year to the (more freely organised) doctoraal phase. Is there a risk 
of students falling behind in this transfer? 

the structure of the curriculum: its coherence, the order in which the 
subjects, etc are taken; depth as opposed to breadth; 

the intensity the programs; are the programs so set up that the 
average student can complete them successfully in the available time? 
What is the situation concerning drop out and delay? 

the manner in which the curriculum has been brought about and the 
responsibility for it; 

the relationship between a broad as opposed to a specialized educa- 
tion; 

the place of the master thesis, work experience, graduation project. 
What place do these occupy in the program? What is the nature of the 
supervision? 

what is the situation concerning student advising, student guidance 
and counselling, recording and registration of student progress? 

articulation between educational program and society; employment 
prospects for graduates; 

education and research as warp and weft. Are the latest scholastic 
developments integrated into the educational program? In what ways do 
students come into contact with research? What role does research play 
in the educational program? 

the internal quality control. Is there a good evaluation system? What 
is done with the results it produces? 



5.3 The preparations Bade by the visiting committee for the visits 

The visiting committee should meet together as soon as possible after it is 
appointed. At the first meeting the VSNU gives further information concer- 
ning the review process and indicates what is expected from the committee. 

In the latter part of the day the members of the visiting committee 
exchange ideas concerning the task, how to make explicit topics to Oe 
raised and the distribution of the topics among the committee members. 
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Further it is established what information remains to be asked for. The 
available time is also used to set up a draft timetable of the visits (with 
the assumption that there will be one visit every two weeks). 

After the first meeting a further meeting is arranged by the chairperson 
and secretary of the committee with the members of the VSNU-Discip Unary 
Board to which the degree program belongs. The coordinators appointed by 
the faculties are also invited to this meeting. This meeting is intended 
for the further discussion of the task and the committee's conception of 
it At the same time points to which the committee should pay particular 
attention can be put forwarr Further the draft time table can be sub- 
mitted. The arrangements wh^h may be agreed to at this time 
confirmed in writing by the chairperson and secretary as soon as possible 
and conveyed to the faculties and the committee members. 

The faculties have to present the self study at least one month prior to 
the first visit of the visiting committee. After consideration of the sell 
studies the visiting committee meets together in order to discuss them. The 
committee analyses the self studies and on the basis of the different 
elements formulates the questions which they will pose. Where a committee 
is concerned with several degree programs in its remit then, on the basis 
of the self studies, it is decided how best the various degree programs can 
be approached. On the basis of the self studies the committee puts forward 
a draft program. 

5.4 Formulation of the program 

The secretary makes contact with the coordinator in the faculty as soon as 
possible after the last preparatory meeting. Logistical arrangements are 
made in broad outline. At the same time further appointments are made 
concerning the visit and the secretary sets up a program for the visit in 
consultation with the coordinator. 



S.5 The visit 

In principle the visit to a faculty lasts for two days. Previous committees 
have found it most useiul to meet together on the evening prior to a visit. 
This evening can be used in order to meet the chairperson of the Executive 
Board of the university and the chairperson of the Faculty Board. The 
chairperson of the visiting committee acts as host and invites the 
of the board either to an informal dinner or to a short meeting at which 
the committee gives a short introduction concerning the committee s 
conceptions of its task and its manner of working. At the same time the 
opportunity is used to go through the timetable once more with the faculty 
coordinator. 

In the course of previous reviews the custom has grown up for the Faculty 
Boards to act as hosts on the evening of the first day and to invite the 
members of the committee to dinner. 
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The plan of the program is left to the judgment of the committee. Ex- 
perience^ suggests however that the visit can best be commenced by discus- 
sions with "embers of the faculty board and the author(s) of the self 
s"-udv After that, there should be discussions with members of the degree 
program committee, followed by discussions with lecturers, students, study 
coordinators and so forth. Should it be deemed necessary as a result of 
these later discussions, a second discussion can possibly be set up with 
the faculty board and / or the degree program committee at the end of the 
second day. 

The visiting committee should also exchange views with groups of students 
ffor the involvement of students with the program review see also 6.6). 
There wili thus be discussions with a group of about ten first year 
(propaedeutic) students, a group of students from the doctoraal P*«e 'also 
about ten), and with the Ph.D. students (AIO's). The composition of the 
Ituden group requires special attention. The panels should be assembled in 
cooperation with the coordinator and the secretary. Where possible the 
study or student associations should be brought in. 



5.6 The report. 

Taking into account the variety of the functions that a visiting committee 
fulfils (to be summarized as giving feedback to the faculty and the 
furnishing of a contribution in accountability concerning the quality of 
tne educational program), careful consideration must be given to the 
reporting. A distinction has therefore to be made between: 

Thg Verba l Report 

The visiting committee will in any case report their findings verbally at 
the end of each visit. This will for the most part be directed to the 
giving of feedback to the faculty / degree program. As many of the people 
concerned as possible should be present at the giving of this feedback: the 
Execute Board of the University (CvB), Faculty Board, Degree program 
Committee and so forth. The verbal reporting should be vie«ed « a first 
reaction to the main points and not as a carefully thought out. all 
inclusive judgment from the committee. 

If several degree programs are involved in the same review, then tfcs 
cLittee al wi ii 8 ju8t »ake one verbal report to the faculty board and will 
not make a separate report to each degree program. 

Tbp Written Report 

On conclusion of .11 the visits the visiting committee brings out a report 
that is presented to the Governing Board (Bestuurlijk Overleg) of the VSNU. 
The Council makes the report public. Besides providing feedbac k this 
report sets out a balance sheet representing the state of affairs within 
the university concerned and in the discipline / degree program viewed as a 



whole. 
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Considering its public nature the visiting committee is asked to arrange 
the report according to a specific model. The contents will be arranged in 
broad outline be as follows: 

Introduction: the remit and the terms of reference of the committee; the 
composition of the committee 

Chapter 1 the findings of the visit to the faculty X of university X, 
if necessary further subdivided into: 

- the findings in degree program a; 

- the findings in degree program b etc. 

Chapter 2 the findings of the visit to the faculty Y of university Y. 
if necessary further subdivided into: 

- the findings in degree program a; 

- the findings in degree program b etc. 

A general overview of the discipline / degree program in its entirety 
precedes or follows the findings per faculty. General conclusions and 
recommendations will be included here. The report will end with a short 
evaluation of the way the review progressed. 

For each faculty the reporting will be presented according to the same 
pattern. In any case the following topics will be dealt with: 

1 general description of the degree program visited 

This section has as its goal the portrayal of the degree program for 
an outsider. There will be a description of the program and the 
specializations in which students may graduate in very general 
terms. Inclusion of quantitative data: numbers of students, pass / 
fail percentages, staffing and so forth. Further, a description of 
the organizational situation of the degree program. 

2 Ttlff nhj actives 

Have objectives been formulated? What do they comprise? How do they 
relate to objectives formulated elsewhere, and how do they relate to 
the expectations of the employment field (how does the degree 
program portray itself?) How are the objectives translated into 
final assignments? And how are they reflected in the curriculum? 

£ Wrtimational design u nd the ed»r«tional process 

The development of the programs (first year (propaedeutic) and 
degree (doctoraal) phases): what graduation options are there? Are 
they set up according to educational principles? Is completion of 
the program in the prescribed time feasible? What is the student 
success rate? 

Are there any particular bottlenecks in the program? The selective 
and orientation functions of the propaedeutic year. The master 
thesis, work experience stage and / or practicals; are they bottle- 
necks or not? What is the articulation between first year (propae- 
deutic) and degree (doctoraal) phases? How are students assessed? 
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4 Educationa l organization 

Is there any indication of a formulated educational policy7 In what 
vays is it put into effect? What role do the various committees 
play7 Are faculty and institutional policy consistent? 
Personnel policy $ study load; the allocation of staff to the 
different phases. • 
Student progress, guidance and counselling and the advice function. 
The material restraints of the working environment. 

£ The internal quali ty control 

What is done internally in terms of quality control7 Are there good 
evaluation systems in place? What is done with their results 7 Are 
the students involved in evaluation procedures? Opinion concerning 
the self study which has been carried out. 

§_ Graduates 

Are the objectives of the educational program indeed achieved? Do 
the graduates fulfil the requirements made of them by the profes- 
sional field7 Is there any indication of definite professional 
profiles? Is it known where the graduates are taken up? Is there 
substantial graduate unemployment? 

2 JpfpmatioTiin comparisons 

Are there international contacts? Is there any indication of student 
exchanges and so forth? Hov does the subject stand within a compara- 
tive international framework7 

£ strengths / weaknesse s analysis nf the degree program 
£ Conclusions and re commendations 



In the treatment of the various topics a clear distinction must be .made 
between description and opinion, The following arrangement can be borne in 
nind for each topic: 

Topic 

a. Description of the situation, such as for example some topic which is 
brought up in the self study. 

b. What is the view expressed in the self study7 

c. What are the findings /opinions/ judgment expressed by the committee? 
d : Recommendations . 

The points summarized must in any case be dealt with. The framework put 
forward does not prevent a visiting committee adding further points to be 
raised, should that be desired. 
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Before the visiting committee presents its report to the Governing Board 
(Bestuurlijk Overleg) , a draft version (of those parts which are the 
concern of the faculty) as well as the general section should he sent to 
the Faculty Board with the request to amend any factual errors. A copy is 
also sent to the Executive Board of the University (CvB) . 

If no difference of opinion occurs with respect to any factual inaccuracies 
which may arise, then the committee should adapt the report to take these 
points into account. But the borderline between the opinion of a faculty 
concerning 'factual inaccuracies' and 'coming to an opinion other than that 
of the committee 1 cannot always be defined sharply. Should a faculty wish 
to do so, a reaction from the faculty can be included in the report as an 
appendix. 
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6.1 Faculty or degree program 

The "discipline directed design of the program review does not always run 
parallel with the organisational placement within the university of the 
degree program concerned; sometimes a degree program is also a faculty, 
sometimes a subject team (vakgroep). Usually several subject groups are 
concerned with the implementation of the educational program of the degree 
program concerned. It is the degree program X, in whatever organizational 
context it may be set, which is attended to by the visiting committee. And 
it is the Board of the Faculty under which the degree program falls which 
is responsible for the preparation and running of the visit. Should there 
be any question concerning what is expected from a faculty within the 
context of a review, then it is the activities of the degree program to be 
visited under the auspices of the faculty which are meant. 

6.2 The coordinator 

It is set down in article 3.1 of the protocol that, as soon as the faculty 
has been informed of the anticipated review, a coordinator is appointed. 
From the reviews that have taken place to date it would appear that certain 
criteria need to be taken into account: 

it is irsportant that the coordinator to be appointed should have good 
relationships with the faculty and degree program as a whole, with the 
Board of the Faculty as well as with individual lecturers; 

it is important that the coordinator to be appointed should have easy 
access to the Bureau of the Faculty; 

the coordinator must be able to set up arrangements with the visiting 
committee vith authority. 

Since the coordinator will be responsible for the smooth running of the 
review it is advisable for the coordinator also to be involved with (the 
implementation of) the internal evaluation which leads to the self study; 
and it is also advisable to have the coordinator be chairperson of the 
evaluation / self study committee. 

The coordinator is also the person with whom the Secretary of the visiting 
committee will take up contact in order to arrange the program for the 
visit. S/he is also contact person for the visiting committee and, should 
it be necessary, for the staff member of the VSNU bureau who is concerned 
with the external quality control (see 7.2). 

Finally, the coordinator has an important part to play in (seeing to) the 
communication of information to staff and students (see 6.6). 
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6.3 Information neceasary for the purposes of the visiting committee 

The most important information for the visiting committee is the self 
study. In addition to that, the faculty is expected to send the Study Guide 
to the visiting committee. Ten copies of the self study and the Study Guide 
are sent to the Secretary of the committee or to the VSNU coordinator. The 
educational report of the faculty has yet to become a universe I tx. 
Should there be such in the faculty concerned, copies of the most recent 
report should also be sent. 

The visiting committee can let it be known should supplementary information 
be required, or that it should be made available for their perusal on the 
soot This is a matter of recent evaluation reports of (parts off) tne 
program, faculty development plans, examples of master theses, lecture 
notes photocopied articles, lecture handouts for students and workbooks 
prepared^ for student use. The faculty decides with the Secretary which 
materials should be sent, and which should be made available for perusal at 
the time of the review. Where for example examination material or naster 
theses are concerned, these materials need to be handled confidentially 
and, should they be sent to the review, they have to be carefully returned. 

6.4 The self study 

The setting up of the self study is discussed in detail in Chapters 3 and 
4 It is no doubt superfluous to point out once more that it is the seir 
study, based on an internal evaluation, which forms the hinge linking the 
internal and the external quality control. The greater the extent to which 
internal evaluation becomes a regular activity within the «J 
writing of a faculty educational report becomes a habit, the less will be 
the intensity of the activity demanded for the setting up of a self study. 

It is inevitable that, in the writing of the self study, there will be some 
conflict between tactics and candour. There will be a temptation to make a 
•public relations' document of the self study. This will only have a 
negative effect on the smooth running of the review, since the committee 
will be obliged to spend time cutting through the trappings, and that will 
be at the cost of the time available for a dialogue. 

Since the report of the visiting committee is public, and the committee 
will refer to the self study, the visiting committee will consider the self 
study to be a public document. 

6.5 Preparation for the review 

* Meeting for the faculties concerned with the review 

In order to prepare for the visit, the VSNU organizes a meeting for all the 
degree programs concerned as soon a. possible after the educational visit 
is formally announced. It is of the utmost importance that both staff and 
students in the faculty should be properly informed of the purposes and the 
progress of the review. The coordinator will have to pay particular 
ittf .tion to this. The meeting is primarily directed to the provision of 
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further information concerning the visit and what the expectations are. The 
time is really too short to go more deeply into the manner and style of the 
self stuoy For this the Centre for Studies in Kigher Education Policy in 
the University of Tvente (CSHOB) nay be drawn on. 

A good vay to ascertain the aims and procedures of the review is to contact 
a faculty which has already been evaluated. 

* Contact between Secretary and Coordinator 

The Secretary of the visiting committee will contact the coordinator as 
soon as possible in order to discuss the logistic arrangements necessary 
for the visit. A program is set up in consultation with the Secretary. 

6.6 Student involvement 

The involvement of the students in the review process is of the^ S r "test 
importance. Their opinion concerning the quality of their educational 
provision needs to be taken up by the visiting committee as care fully as 
possible. The faculty must see to it that the students are veil inf ormed 
about the review and are actively involved in it. The contribution of the 
students can take place in a number of different ways: 

Through a good system of internal quality control the participation of 
students is ensured in the established student evaluations of their 
educational provision. The presence of these and the results should be 
taken up in the self study. 

It is advisable also to involve students in the self evaluation. 

- In the course of the visit, discussions will take place with the 
faculty board and the degree program committee, in both of which 
students are also represented. *._«.,. 
Further to that, discussions can take place with groups of first year 
and doctoraal (first phase) students. ^„ aA «. n 
The study association within the degree program must be encouraged to 

participate in the review. * < e«„* M .«.i- 

- Should the students consider that their voices are not sufficiently 
attended to in the self study, they should be offered the opportunity 
of putting their own evaluation into writing for the purposes of the 
visitiig committee. 

6.7 The visit 

The visiting committee proposes a timetable for the review and puti ( this 
forward for the faculties. Only the most weighty objections can lead to the 
alteration of the date of the proposed visit. 

For the procedures during the visit see 5.5 

With respect to the program it needs to be said that this is set up in 
consultation between the visiting committee and the faculty. 0nce 
program has been set up. then the visiting committee may only diverge f rom 
it for the most compelling reasons. It is however established m the 
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protocol that it is ultimately the visiting committee which determines the 
program and decides with whom discussions will be held. 

Whatever costs are incurred by the faculty for the preparation and servi- 
cing of the visit are to be borne by the faculty, with the exception of the 
travel and accommodation costs of the committee members. 



6.8 The report 

For the procedures concerning the report, see 5.6. The draft version of (a 
part of) the report is sent to the Faculty Board and the Executive Board of 
the University. From the faculty it is expected that any factual inac- 
curacies will be reported to the committee within 14 days. 

Should the faculty wish to react to the report other than to point out 
factual inaccuracies, then this is brought to the attention of the commit- 
tee. The faculty ought to make it clear if it is their wish that the 
reaction should be included in the final report as an appendix. 



6,9 The follow up to the review 

What finally happens with the results of the reviews is in the first 
instance a matter for the faculty and university concerned. It has however 
been agreed in the HO-Kamer (Higher Education advice committee to the 
Ministry) that the universities should make clear in their educational 
development plans and their educational reports what has or will be done 
with the recommendations. 
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7. THE VSNU AND THE QUALITY CONTROL 



7.1 The timetable of the review 

The review system has been set up following s joint decision of the 
diversities in the Netherlands. It is the Governing Board (BO: Bestuurlijk 
Overleg) of the VSNU which commissions a visiting committee. 
The Governing Board has set up a timetable of reviews for the period 1988 - 
1992 (see article 4.3 in the protocol). Reference should be made to the 
Publication -Rooster voor ondervijsvisitaties en indeling der «tudierich- 
?£gen near visitatiecommi.sie- (Timetable f^™ 1 *™*"!^^ 
the distribution of degree programs among visiting committees) (VSNU, July 
MM) for the timetable and distribution of the degree programs among the 
visiting committees. 

In contrast with the timetable 1988-1992, which assumed a five year cycle, 
Se ^SSblTtat now been amended to spread the visits for 1992 over 1992 
iXHn orS" to set up a six year cycle (see the timetable in appendix 

5). 

The faculty is formally apprised of the impending review in about May of 
the Preceding year, and the VSNU-Disciplinary Board is requested to put 
rorwa P rd namts 8 for the visiting committee. The period f rom 1 «ay to December 
is then available for the implementation of the self study (although it 
transpires that many faculties commence that earlier). 

The actual visits can take place in the following year. 

An endeavour is made to arrange for visits to take place every two or three 
Teels, so that the complete review (including the reporting of it) can be 
completed in about six months. 

7.2 Responsibility of the VSNU 

The Governing Board of the VSNU commissions the program reviews. The Bureau 
of the vTnU is responsible for their coordination. A process coordinator 
has leer appointed within the bureau of the VSNU for that p«P^.i»J 
other tasks! he is responsible for the preparatory activities for the 
reviews, the* notification of the faculties, the direction of the visiting 
committee and the guidance of the secretaries. Should there »>e any ques- 
tions which a faculty or visiting committee wishes to put concerning the 
review process, then contact should be made with the process coordinator. 

7.3 Contact with the Inspectorate 

From the side of the VSNU there is regular consultation withjAe Inspec- 
torate concerning the progress of the external quality control. The 
inspectorate has the task of a meta-ev.luation of the program 
is to say. an evaluation of the manner in which the program review has 
taken place. 
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7.4 The Appeals Committee 

Should any problems arise in the course of the reviews, such as differences 
of opinion between the visiting committee and the faculty, then both the 
faculty and the visiting committee have the right to request the The 
Governing Board of the VSNU to set up an appeals committee that has the 
confidence of both parties. The appeals committee operates immediately. 
Should the committee be unable to resolve the difference, then it is 
decided by the The Governing Board. 
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PROTOCOL 
FOR 

EXTERNAL QUALITY ASSESSMENT 

UNDER THE RESPONSIBILITY OF 

THE ASSOCIATION OF UNIVERSITIES IN THE NETHERLANDS (VSNU) 
Laid down by the Governing Board on 4 November 1988 
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PREAMBLE 

The reason for setting up a system of external quality assessment is given 
in the nolicy document Higher Education: Autonomy and Quality (HOAK-nota, 
Ministry of Education and Science. 1985). in which the universities were 
granted greater autonomy and greater freedom in designing programmes. The 
minister pointed to the need for a guarantee of the quality of education 
and for the monitoring of quality. To this end. the 1986 University 
Education Act provided for an Inspectorate for Higher Education. In April 
1986 in the consultations between the minister and the institutions ot 
Higher Education, the conclusion was reached that quality monitoring is. in 
the first instance, the responsibility of the institutions . They would 
therefore have to provide a coordinated external system of quality monito- 

This* led the joint universities to set up a system of external quality 
assessment. 

In th* draft proposal for the Higher Education and Scientific Research Act 
(Sinistry of Education and Science. 1988). article 12 states: -The Execu- 
tive Board of the institution shall be responsible, as far as possible in 
collaboration with other institutions, for providing regular assessment, 
carried out in part by independent experts, of the quality of the work of 
the institutions. In so far as these assessments are carried out in part by 
independent experts, the findings shall be public". 

A system based on committees of external experts was tested from September 
1987 to September 1988. During the trial, a protocol drawn up by the HOAX 
Steering Committee (HOAK-stuurgroep. 1986) was tested. After evaluation of 
the project (VSNU. 1988). it was decided to continue the system of external 
quality assessment, to which end the following protocol was laid down 

A discipline-oriented approach was selected for the trial so that it would 
be possible to link up at a later stage with the monitoring of the quality 
of research being carried out under Conditional Financing. The results of 
the trial gave no reason to depart from the discipline-oriented approach. 
For the time being, external quality assessment will in principle be 
concerned only with education. The protocol therefore refers to quality 
assessment aimed at education. 
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1. THE PURPOSE AND PROCESS OF EXTERNAL QUALITY ASSESSMENT 

1.1. foe purpose 

External quality assessment complements the attention already devoted to 
the quality of education within the institutions. 

The purpose of external auality assessment is: 

a) to obtain, by means of an exchange of ideas between the committee of 
external experts and faculty/discipline, insight Into the quality of 
the education and to contribute to quality management; 

b) to account for the level of education to the outside world. 

1.2. p frr n » d de scription of the Process 

The process of external quality assessment incorporates the following 
phases: 

1. thff preparation: 

. appointment of visiting committee for each discipline; 
. appointment of coordinators in the faculties; 

internal discussion of visiting committee; 

informative contact between committee and faculty; 

Sternal evaluation and compilation of a self-study by the faculty to 
be visited. 

2. tfafi visits: 

. the visiting committee makes a two-day visit to the faculty concerned; 
. the discussions are based on the self-study that has beer, submitted; 
the visit is concluded with a verbal report of the findings. 

3. conclusion: 

. after its final visit to the discipline, the visiting committee draws 
up one report giving a) a general picture of the state of affairs in 
the discipline as a whole and b) the findings in each faculty; 

. the visiting committee sends the general section of the report and the 
section on each faculty to the faculty board with the request that 
factual inaccuracies be pointed out; 

the visiting committee submits the report to the Governing Board of the 
VSNU as its principal; , . c 

the VSNU makes the report public and brings it to the attention of the 
inspectorate, the minister and others; 
. after the final report is submitted, the visiting committee is dis- 
banded. 
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2. THE VISITING COMMITTEE 

External quality assessment is carried out by a committee of independent 
experts, known as the visiting committee. 

2.1. ThP task o f the viflitiag committee 
The visiting committee's task ist 

a) to form an opinion on the basis of information supplied by the faculty 
and by means of discussions held on the spot about the standard of 
education and the quality of the educational process, including the 
organisation of education and the standard of the graduates; in assess- 
ing quality, the committee must look at the requirements /expectations 
of the student, the faculty /discipline and society, and prospective 
employers in particular; 

b) to make suggestions on quality improvement. 

The findings of the visiting committee are put into a public report (see 
2.6). 

2.2. yh» Member* of the v isiting committee 

A visiting committee has approximately seven members (including the 
chairperson). In principle it is selected on the basis of the following 

criteria: . . 

- the chairperson does not work in one of the faculties to be visited; 

- at least four members are experts, whereby the aim is to achieve a 
distribution between experts who work (or have worked) at the univer- 
sity (whether or not in the discipline to be visited) and experts from 
among potential employers or from professional organisations which have 
an interest in the specific schooling; 

- in selecting experts the aim is to cover as far as possible all specia- 
lisations within the field of study; 

- at least one of the members of the committee is a foreign expert 
(either a Dutch expert who teaches abroad, or a foreigner who is 
familiar with the Dutch situation); 

- the presence of an expert in the field of educational planning and 
educational processes in University Education (a theory of education 
specialist, a study coordinator, a study counsellor or a teaching 
methodologist) is desirable; 

- insight into and experience of the university structure and the deve- 
lopments in recent years must be clearly represented on the committee. 

H.B. Should a discipline consider an international committee desirable 
because of the nature of the field of study, it is possible to select the 
majority of the members from among foreign experts. 

The secretariat of the visiting committee is provided by the VSNU office. 
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2.3. tsmiaailsn mnA ■ppoiTitMrn* ftf the v3 

The members of the visiting committee are appointed by the Governing . Board 
of the VSNU on the recommendation of the Executive Board. The hst 
nomiTationris dravn up after consultation vith the Disciplinary Board (DB) 

Tl^^r^ Vt^ofrhe 
covers the discipline to make a recommendation for the selection of the 

b) S^dfsciP^'board submits a recommendation for the selection of a 
committee, taking the criteria into account (see 2.2); 

c) the chairperson of the disciplinary board sends the list of names 
' (tocludix^a proposal for chairperson) vith a brief description of the 

qualification, of those involved to the Executive Board of the VSNU 

d) the Executive Board gives its opinion of the recommendation and draws 
up a list of nominations; . nf 

e) after the list of nominations has been dravn up. the chairperson of the 
VSNU approaches the nominees; f 

f) the Governing Board of the VSNU gives its opinion of the list of 
nominations and appoints the members. 

it t> Tf the Disciplinary Board fails to put forward a proposal within six 
«L of ^^bein 8 sent, the Executive Board may itself draw up a 
list of nominations. 

2.4. Tb p -orking m»i- w of the vl siting committee 

In principle, a visiting committee chooses its own method of working. It 
must, however, observe the following points: 

^ the visiting committee meets as soon as possible after it is appointed; 
' this meeting- is used, among other things, to formulate basic principles 
considered important in the evaluation of the discipline; 

b) as soon as possible after the meeting referred to ^ aj^tt. Tl.ltl»l 
committee calls a meeting with representatives of the faculties to De 
visited? in the context 8 of the Disciplinary Board or otherwise, in 
which task, working methbd. criteria and points of special attention 
are discussed; 

ci after receipt of the self-study (see 3.3) the committee informs the 
} raculty as soon as possible of the supplementary information relating 
to the subject it wishes to receive from the faculty; 

d) the secretary of the visiting committee gets in touch with the coor- 
dinator in the faculty as quickly as possible to work out the logis- 
tics; 

e) in setting up the visits, the points in 2.5 are taken into account; 

f) reporting occurs in accordance with the guidelines in 2.6. 
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2.S. Th" » lBl * ° f the **-«it<ng committee 

The visiting committee visits each faculty within the discipline concerned. 
The following points are observed in respect of the visits: 

a) the secretary of the visiting committee draws up a schedule of visits 
in consultation with the committee and the faculties; 

b) the visit lasts two days; 

c) the programme for the visit is drawn up in consultation with the 
coordinator in the faculty; in any event there will be dueuision. with 
the faculty board, the department board(s) involved, the discipline 
committee, lecturers in core subjects, study counsellors, study coor- 
dinators/students and research assistants; besides holding discussions 
the committee will also look at areas for practical work, laboratories, 
libraries; 

d) the visiting committee decides on the final programme and decides who 
it wishes to interview; 

e) the faculty provides the facilities during the visit (meeting rooms, 
lunches, etc.); 

f) if a committee members works in the faculty to be visited, he/she does 
not take part in the visit; 

a) at the end of the second day, the committee gives a verbal "port of 
its findings to the faculty board. The committee gives the Executive 
Board of the university the opportunity to be present at this meeting. 

2.6. The report 

After all visits are concluded, the visiting committee issues one report. 
It will contain chapters on at least the following aspects: 

1. description of how the visitation process went; 

2. the findings in respect of the aspects referred to under 2.1.a for each 
faculty; 

3. a general evaluation of the discipline as a whole. 

The following procedure is observed in publishing the report: 

a) after all visits in the discipline concerned are completed, the visi- 
ting committee draws up a draft report; the visiting committee sends 
the general section and the section relating to the faculty to the 
relevant faculty board with the request that factual ^accuracies be 
pointed out; these are to be brought to the attention of the committee 
within 14 days and corrected by the committee; if a faculty wishes, a 
reaction from the faculty may be included as an appendix to the report; 

b) the committee submits the definitive report to the Governing Board of 
the VSNU; 



Guide for External Prograe Review 

VSNU, December 1990 S1 



50 



c) the Governing Board of the VSNU decides hov the report will be publish- 
ed and sends the report to the faculty board, the Executive Board of 
the university and other parties involved, and to the minister and the 
Inspectorate of Higher Education for their information. 

2.7. »nd rf j«hur«e»pnt of expenses 

a. The costs incurred by the committee during the visits will be paid 
directly by the VSNU. These include: 

- the costs of the reception dinner on the first evening, 

- the costs of accommodation for the committee members, 

- where necessary, the costs of hiring a meeting room outside the 
institution for the committee's use. 

b. The members of the committee submit an account through the "cretary to 
the VSNU of their travelling expenses and other expenses of their stay 
incurred as a result of their vork for the committee. 

c. All other costs incurred as a result of the visit are borne by the 
faculty concerned. 

d The chairpersons and members of the committee receive a fee for the 
work from the VSNU for each visit to be mads. This fee is fixed before 
the vork commences . 

2.6. pj ffbima«ent; of the visiting CQgmjttee 

The visiting committee is disbanded after the definitive report has been 
submitted. 

3. THE FACULTY 

3.1. coordi nator in thfi facultY 

As soon as the faculty has been formally notified that it will be involved 
tha^ year in a visitation by a committee of experts, it appoints a coor- 
dinator. The tasks of the coordinator may include: 

* organising the self-study (see 3.3.); 

* actina as contact person for the committee and the VSNU; 

* consulting with the secretary of the visiting committee on drawing up 

* SS.SS'S .raffV/atVdents (or arranging for them to be informed) 
about the purpose and process of the visitation. At ^ ¥ , mt . rt . 

In respect of matters relating to the visitation process direct • ^tact 
may be maintained with the coordinator. The coordinator is also the 
"Lessee for .11 correspondence relating to the ^J*"* P 1 ^* 
However, a copy of all official documents must also be sent to the Execu- 
tive Board of the university. 
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3.2. Tnf omtion *«r visiting committee 

a) Prior to the visit by the visiting committee, the faculty must supply 
the following information: 

1) a self -study (see 3.3.). ten copies; 

2) the most recent prospectus, eight copies; 

3) if available, the most srecent annual report on education, ten copies. 

b) The committee may also ask for additional information. 

c) During the visit, the following information must be available for the 
committee's inspection: 

- annual reports, development plans and reports on education, in so far 
as they exist; 

- recent internal evaluation reports on education; 

- a selection of the lecture notes, readers and textbooks used; 

- examples of master thesis and examination papers. 

3.3. Th» «elf-«tttdv 

a) The primary source of information for the visiting committee is the 
self-stud* produced by the faculty, which is compiled in accordance 
with the guidelines given in Cba: er 5. In so far as the annual report 
on education conforms to these guidelines, it may be submitted in 
place of the self-study. 

h) The self-study contains both a description and an analysis of existing 
problems and an outline of the plans to tackle these problems. In any 
event it must contain information on: 

* the student intake 

* course hours and course progress 

* programme content 

* educational process 

* programme organisation/management 

* the graduates. 

c) The self-study must be submitted by a date to be fixed by the visiting 
committee. In general, this will be at least one month before ttoe date 
of the visiting committee's first visit to a faculty. 

d) The faculty board is responsible for issuing the self -study. 

e) The self-study is the committee's point of departure for the discus- 
sions during the visit. This means that everyone involved in the 
discussions is assumed to be familiar with the contents of the report. 
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4. THE VSNO AND EXTERNAL QUALITY ASSESSMENT 

4.1. Thf rf . Bp0 n B ifr n<fci» s of the VSNU 

a) The Governing Board of the VSNU is the visiting committee's principal. 

b) The VSNU office is responsible for coordinating external quality 
assessment. 

4.2. aPlffis7« eo—ittee 

In the event that problems arise in relation to the visitation or disputes 
occur between the* visiting committee and a university or faculty the 
Governing Board will appoint an appeals committee. Each of the parties can 
brlng^e dispute befor? this committee. If the appeals committee is unable 
to resolve the issue, it will refer it to the Governing Board of the VSNU, 
which will make a binding decision. 

4.3. Th» trinitaMtm schedule 

The Governing Board has set up a timetable of reviews for the period 1988 - 
1992 (see appendix 4). Reference should be made to the publication "Rooster 
voor ondervijsvi.itaties en indeling der studierichtingen naar 
commissie" (Timetable of External Program Review and the distribution of 
degxte programs among visiting comaittees) (VSNU, July 1989) for the 
timetable and distribution of the degree programs among the visiting 
committees • 

S^ases not provided for by the protocol, the Governing Board of the VSNU 
will decide. 
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Survey of visits 1988-1990 



vc 


number of 1 
fields of study 1 


number of 
faculties 


1988 






p6ychologie 


1 1 


7 


physics 


2 1 


9 


astronomy 






history 


1 


7 


mechanical 
engineering 


2 


4 


1989 






geography 




7 


non-ve stern etc I 


*l /. 

1h 


6 






10 


mathematics & 






information theory 










2 


industrial design-' 


2 




ing t aviation- 






and space travel 






1990 






biology 


1 2 
1 


7 


lav 


7 


9 


philosophy 


2 


9 


economics 


8 


1 *7 
1 / 


electrical 
engineering 


2 


3 


total 


1 50 


1 87 
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TABLES FOR aUAHTITATIVE DATA 

a h.i«« thould b« elaed at between data which art available at a central lavel (Board of Adainistra- 
tiSt^d^U which !rt available at a daeaotr.li.ed level (faculties, dep.rt.ents) . This is the only 
the co^raoU ty? reliability, .nd verif lability of the d.ta. therefore it i. necessary 
tZt the^.wltS. n^£d' "he s.ae d.ta (the sate d.tu. date, the ...e definition, the MM maber) li 
!w! ' ..."!. / .nri uMrh are D rov<ried to exter n *! institutions bv the central 
^ff ^V "!^ to a faculty (and /or a deparUp, the 

■Sftf ^mSeM etota (dwSSt™™^. dlf fBTint datu. date, etc) Is advisable, those data could 
£ Jjpf . iSJffi Tthe cause of the differences, with respect to the 'central' data, are 

l yjTS ^r^'dlt: ^IsT^ 1 ]^ t^hose which .r. Included in the ^ 

smias partly with a view to external provision of inforaation. e.g. the C B.S. (Central Bureau of 

sm^ics^D^Xr count, tha scientific raport. the educational report, the financial account. 



Tabla 1: Influx of first-year students (counting date Deceaber 1) 





Fulltiae 


Pa 


rttiae 




year of study 


N 


F 


total 


M 


F 


total 


1982/1983 














1983/1984 














1984/1985 














1986/1987 














1987/1988 














1989/1990 














ttc. 















f^S^lSLu are the Inflowing students who are aatricul.ted at the Ifisultv. for the first ti.e. 
i.e. including students, coaing froa other faculties. 
The counting data is Deceaber 1, as with the CBS count*. 

Table 2: Total nuaber of students (counting date Deceaber 1). 





Fulltiae 


Pa 


rttiaes 




year of study 


M 


F 


total 


M 


F 


total 


1982/1983 














1983/1984 






4> 








1984/1985 














1986/1987 














1987/1988 














1989/1990 














etc. 
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size of 


I 

X P-exaas after: 


1 

X D-exaau of the P-1nflux after: 


X D-exa«s of the 


D-influx a 


1 

fter: 


year of 
study 


cohort 
on 

dec. 1*) 


1 year 


2 years 


>2years 


4 years 


5years 


6years 


> 6 years 


4yrs . 


5yrs . 


6yrs. 


> 6yrs. 


1982/83 






••) 




*•*) 
















1983/84 


























1984/85 


























1985/86 


























1986/87 


























1987/88 


























1988/89 


























1989/90 


























etc 



























**\ the percentages are cuawlative 4 

the first percentage has been calculated on the basis of the first-year influx (see eoltwt 1) 
second nuaber has been calculated on the basis of the doctoral influx. 



category 


H 


F 


total 


percentage 
of 

graduates 








persons 


fte's 


Professor 












Principal 

University 

Lecturer 












University 
Lecturer 












Remaining 

scientific 
personnel •) 












Assistants in 

Training 












total 













*> excluding A.I.T., but Including students-assistants. 
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APPENDIX 3 



CONFIRMATION OF APPOINTMENT 

The Governing Board, 

duly obeying the protocol for externel quelity ....«..*«. Lid down on 4 "ov-ber 1988. .nd the _ letter to 

the Executive Boerds of the universities deted reference in which the visitetion of 

wee formally announcd, 

DECREES: 



1. A visiting coeertttee for 1« appointed 

2 1* appointed chairperson and 

aeeber of the visiting committee 

The following are eppointed members of the committee: 



is appointed secretary of the visiting committee 



3. The tesk of the visiting eosarittee is: 

a) to for. an opinion on the basis of information supplied by the feculty end by leans of discussions held on the 
Sot .Sou? X mnderd of eduction end the quelity of the educ.tional process, including the org.ms.non of 
&ta?« I st«J.rd of the gr.du.tes; in ...es.ing quslity. the cc-ittee «t lock .t the recxiir^t./expecftiens 
o\ the student, the feculty/discipline end society, in perticuler prospective employers; 

b) to aeke suggestions on improving quelity. 

The findings of the visiting committee ere issued in e public report. 

4. In cerrying out its tssk the visiting committee will in any event include: 

a. the discipline in the feculty et the university of 

b. the discipline in the feculty et the university of 

c the discipline in the feculty at the university of 

d! the Si.cipline in the feculty et the university of 

«. the discipline in the feculty et the university of 

f . the discipline in the feculty et the un ver.ity of 

g. the discipline in the feculty et the university of 

5. The visiting committee will decide on its own working method, taking into eccount the guidelines in the protocol. 

6. The visiting committee will submit its report tc the governing Boerd one month after the final visitetion. in 
any event before 198.. 

7. The costs Incurred by the committee during the visits will be paid directly by the VSNU. 

?£ -i-bers of the committee submit en eccount of their tr.yelling expense, end other expenses of their st.y 
incurred as a result of their work for the committee, through the secretery to the V5NU. 
ThTch.1r £.<x [ eiULr. of the cc-1ttee rec.1v. . fee for the work fro. the VSNU for e.ch visit to be «de. 
This fee 1s fixed before the work cc m r nces. 
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S. Copies of this confirmation of appointment will be tent to: 



. the chairperson, «e«bers and secretary of the committee; 

- the governing bodies of the universities 

: s: SJSrSi; srssu. — 1^ *. » ^«k., 

- the Inspectorate of Higher Education; 

- the organisation where the ■cabers are employed. 

Utrecht, «• 

(P, van der Schans) 
chairman VSNU 
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TMETAKL Of KVXEVS 1991-1996 
1991 

* Western Languages 

* Theology 

* Ceology 

* Political Science ♦ Public Administration 

* Medicine 



* University Teachers Education 

• Pedagogics ft Didactics 

* Sociology ft Non-Wester Sociology/cultural Anthropology 

• Agricultural Sciences 



* Experimental Progress In Linguistics ft Literature 

* Chemistry ft Pharmacy 

* C1v1l EnQlneerlng, Architecture, Geodesey 

* Dentistry 



* History, Arts history ft Archeology 

* Psychology 

* Physics ft Astronomy 

* Business studies 



* Mathematics ft Information Technolgy 

* Geography, Plenning, Prehistory ft Protohistory 

* Non-western Languages ft Literature 

* Mechanical Engineering c.a. 

1996 

* Philosophy 

* Biology 

* Electrical Engineering 

* Law 

* Economics 



1992 



1993 



1994 



1995 
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PUBU KATIES IN NET KADER VAN DE KUALITEITSZORC 

. vwo (19M) OVER DE KUALITEIT VAN NET NEDERLANOS UNIVERSITAIR ONOERUIJS. Utrecht: VSNU. 
. vshu (1W 8b). DE EVALUATIE VAN HET PROJECT PROEFVISITATIES. Utrecht: VSNU. 

- VSNU (1988c). DE VSHU-VERKEHNINGSCOHHISSIE-PROTOCOL. Utrecht: VSNU. 

- VSNU (1988d). CIDS VOOR DE EXT ERNE KUALITEITSZORG . Utrecht: VSNU. 

- VSNU (1988.). GUIDE TO EXTERNAL QUALITY ASSESSMENT. Utr«cht: VSNU. 

- VSNU (WW). ROOSTER VOOR ONDERUUSVISTATIES EN INDELIN6 DER STUDIERICHTIHGEN NAAR VISITATIECOHNISSIE. Utrecht: 



VSNU. 



VSNU vl.it.tieco-L.ie Seogr.fie (1990). RAPPORT UAARIN DE VISITATIECOHNISSIE 6E0GRAFIE HAAR BEVINDINGEN TEN 

VASTCELECO. Utrscht: VSNU. 

JSoJSSK! "SSiSK.Mll. SLAVISCH, FMWISOI. ..ET-ESIE.S i» AFRIKAANS MSSFT VAST- 
GELEGD. Utr«cht: VSNU. 

z i i _ MQOnx ■APPfiOT UAARIN DE VISITATIECOMHISSIE IO+LR HAAR BEVINDINGEN TEN AAN- 

HEEFT VAST6ELE6D. Utrecht: VSNU. 

- VSNU (1990). KUALITEITSZORS BINNEN HET UETENSCHAPPELIJK OHDERUIJS. Not. voor h.t AB en BO v.n de VSNU. Utrecht: 

VSNU. 

- Acheron, J. A. (1988). QUALITY ASSESSHENT BY PEER REVIEW. Utrecht: VSNU. 

o.—inir t Moooi Proar.M. erkenning in het licht ven voortgeende intern.tion.U.erlng. In: Heijnen, G.U.H. 
' '^SiLSTill ^SZSStSh XVALITEITSZORG. HAARBORG VOOR KUALITEIT IH NET HOGER CTOERUUS. Bromngen: 
COUOG. 

- K.U. H R (1989) THE FUTURE OF SELF-REGULATION IN DUTCH HIGHER EDUCATION. P.per presented et the meting of the 

AeVocietion for University Governance end H.n.ge-ent (VUBW) Utrecht. Twente: Cheps. 

u . . , M9fl ox EXTERNAL QUALITY ASSESSHENT: A BURDEN OR A RELIEF FOR THE FACULTY? Paper presented 

" "™tlTM*)L^™r%i Sd.rruniver.Uy d^rt^f. Groningen. October 23th- 2 7th. 1989. Utrecht: 
VSNU. 

U1JS. Gronlngtn: COUOG. 

- Vro.1jen.tijn. A.I * J. A. Acheron (1990). CONTROL ORIENTED QUALITY ASSESSHENT VERSUS IHPROVEHEHT ORIENTED QUALI- 
FY SKXnT. P.pw pr..ented .t the CHEPS-conferenc. 'Quellty .....»ent in Higher Eduction- . Utrecht lurch. 
16, 1990. Utracht: VSNU. 
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h*t beioek v.r rf r Y i f j.-.H geo .«i«t1ei. Utrecht: VSHU. 

Vsnu (1988). p* «/«lu.t1e yt" bfl PXSlSfit BCfiSMlilltiii« Utrecht: VSNU. 
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